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Citizen science programs provide opportunities for students to help professional 
scientists while fostering science achievement and motivation. Instruments which 
measure the effects of this type of programs on student motivational beliefs are limited. 
The purpose of this study was to describe the process of examining the reliability and 
validity of The Citizen Science Self-Efficacy Scale (CSSES) designed to measure the 
effectiveness of citizen science programs on student self-efficacy for scientific 
observation skills. Fifteen (n =15) field experts and 248 (n = 248) eighth grade students 
participated in three studies. The results suggest that the psychometric properties of 
this scale are sufficient. Implications for the development and utility of self-efficacy 
scales in a variety of citizen science contexts are discussed. The aim of the present study 
is twofold: (a) to establish the psychometric properties of a scale developed to measure 
student self-efficacy beliefs for scientific observations in citizen science programs and 
(b) to describe the process in the validation of a self-efficacy scale to support 
researchers who want to create their own scales for similar citizen science programs. 
Three studies were conducted to develop the Citizen Science Scale (CSSES) and evaluate 
its psychometric properties. The purpose of the CSSES was to develop a measure 
suitable for analysis within a social cognitive career framework and informal natural 
science contexts. The findings in the present study found that the measure had an 
acceptable unitary factorial structure and high internal reliability of .89 for the CSSES. 
The purpose of the Citizen Science Self-Efficacy Scale (CSSES) is to assess individual’s 
beliefs about their capabilities for scientific observational skills. This scale is applicable 
to measuring individual's self-efficacy in outdoor learning contexts (e.g., horseshoe crab 
citizen science context). Given that self-efficacy is a strong predictor of academic 
achievement and motivation, self-efficacy scales like the CSSES may provide a way for 
stakeholders involved in outdoor education to measure student gains and to 
substantiate program effectiveness. From a methods standpoint, the contribution of this 
work is to serve as a guide of how to develop a self-efficacy scale. 
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INTRODUCTION 

An array of research studies have examined the influence of motivational 
constructs such as self-efficacy, task interest, outcome expectations, and goal setting 
as major predictors of science achievement (Britner & Pajares, 2006; Hiller & 
Kitsantas, 2014; Jensen, Scherer, & Schroders, 2015; Lent, Sheu, Singley, Schmidt, 
Schmidt, & Gloster, 2008; Navarro, Flores, & Worthington, 2007; Patrick, Care, & 
Ainley, 2011], Self-efficacy, or an individual's assessment of how well they are 
capable to perform a specific task, is one of the most investigated constructs and the 
focus of the present study. Learners with high self-efficacy are more likely to become 
motivated to engage in behaviors such as goal setting, organization, and help 
seeking, with the ultimate goal of achieving mastery for a specific skill or content 
knowledge (Zimmerman, 2013; Zimmerman & Schunk, 2009], Thus, self-efficacy 
beliefs closely align with self-regulation capabilities and academic progress 
(Zimmerman & Martinez-Pons, 1990], Jointly, self-efficacy beliefs and science 
achievement influence future course selections and science career choices (Patrick, 
Care, & Ainley, 2011], For middle school students, there is a strong connection 
between self-efficacy and career pathways (Hiller & Kitsantas, 2014; Navarro, 
Flores, & Worthington, 2007; Rogers & Creed, 2011], Individuals with high self- 
efficacy are more likely to develop sustained interest for a specific activity (Schunk 
& Pajares, 2005], 

Self-efficacy beliefs may vary across subject domains, task requirements, and 
situations (Bandura, 1997], Within the realm of science instruction, a fundamental 
issue for students is the ability to improve and apply scientific observation skills, 
preferably through self-regulatory strategies and modeling (Hiller & Kitsantas, 
2015], Differentiating from daily observation skills, scientific observation skills 
evolve over time, particularly with the guidance of models and involve activities 
which include practices related to data collection and measurement (Eberbach & 
Crawley, 2009], Methods behind data acquisition ultimately influence 
interpretations of results within scientific studies (Cartwright, 1989], As a result, 
assessing students' self-efficacy for scientific observation skills may provide 
important information about students' science achievement and career motivation, 
particularly for students interested in science occupations (Hiller & Kitsantas, 
2014], The aim of the present study is twofold: (a] to establish the psychometric 
properties of a scale developed to measure student self-efficacy beliefs for scientific 
observations in citizen science programs and (b] to describe the process in the 
validation of a self-efficacy scale to support researchers who want to create their 
own scales for similar citizen science programs. 

Although often omitted from formal school curriculum, citizen science is an 
activity which provides opportunities for heightening self-efficacy, motivation, and 
metacognitive growth in the sciences (Hiller & Kitsantas, 2014; Jeanpierre, 
Oberhauser, & Freeman, 2005; Sutton, 2009; Trumbull, Bonney, & Grudens-Schuck, 
2005], During citizen science programs, professional scientists recruit volunteers to 
collect data on scientific studies which require large scale data collection (Fowler, 
Whyatt, Davies, & Ellis, 2013] including biotic and abiotic factors. Scientists who 
require expansive data sets for their research sometimes rely on the scientific 
observation skills of citizen scientists (Snail, Kindvall, Nilsson, & Part, 2011], 

Adult participation in citizen science programs may assist scientific researchers 
in assessing a range of topics of study including zone coverage, species distributions, 
and habitat factors while maintaining quality in terms of hobbyist data collection 
skills (Crall, Jarnevich, Young, Panke, Renze, & Stohlgren, 2015], Similarly, recent 
studies have highlighted the accuracy of data collection from children and 
adolescents, thereby establishing student data submissions as useful in scientific 
work (Pocock & Evans, 2014], As a result, this type of activity has reciprocal 
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advantages for scientific researchers and students, particularly when students have 
access to field experts as models and mentors. 

By including students in field work, large data collection is possible, which 
research suggests has positive implications for school aged children's self-efficacy, 
career motivation, and achievement (Hiller & Kitsantas, 2014], This type of 
engagement aligns with recent career literature which cites the influential role of 
mentors in establishing career self-efficacy (Day & Allen, 2002], For this reason, 
studies which highlight student participation in field work experiences, require 
well-designed, context related instruments to assess student self-efficacy beliefs. 

A current gap in assessments of STEM education is the limited number of 
measures which assess student learning (Harwell, Guzey, Moreno, Moore, Philllips, 
& Roehrig, 2015], Similarly, as self-motivational beliefs, including self-efficacy, 
directly correlate with academic achievement, self-motivation scales become 
essential in analyzing STEM education benefits, particularly in terms of career 
motivation. Bandura (1997] initially described the contribution of self-efficacy in 
increasing self-regulation, motivation, and academic performance and espoused the 
development of self-efficacy scales related to specific domains. Self-efficacy is 
central in assisting an individual to set goals and strategies to become a self- 
regulated learner. 

Figure 1 illustrates the role of self-efficacy within a cyclical self-regulatory 
feedback loop with the focus of scientific observation skills based on Zimmerman's 
(2000] self-regulated learning model. 

The forethought phase, divided into two sub processes of task analysis and self- 
motivational beliefs, provides the initial support for individuals to attempt a task. 
Once an individual engages in the forethought phase, they will attempt the task in 
the performance phase. Students engage in both self-control (self-instruction, self¬ 
imagery, attention focusing, and task strategies] as well as self-observation (self- 
recording and metacognitive monitoring]. In the self-reflection phase, individuals 
will judge their performance in the sub processes of self-judgment and self-reaction. 
With diminished self-regulatory skills and weak motivational beliefs (e.g., self- 
efficacy] in the forethought phase, individuals may attribute insufficient 
performance on luck or a weak instructor rather than on the type of strategies 
selected for task performance. Ill structured strategies and negative motivational 
beliefs in the forethought phase hinder progress in the next two phases in terms of 
student initiative for learning (DiBenedetto & Zimmerman, 2013], In addition, a 
learner may engage in behaviors such as procrastination in order to avoid starting 
the cyclical process again (Cleary & Labuhn, 2013], As a result, developing strong 
self-efficacy perceptions bolsters proactive self-regulatory behaviors; strongly 
corresponding with motivation, and achievement. This process of guiding students 
through a task, such as the improvement of scientific observation skills, is one way 
to provide opportunities for students to increase mastery experiences which is a key 
source in the development of self-efficacy (Hiller & Kitsantas, 2015], 

With citizen science experiences as an opportunity for collecting scientific data, 
an individual's self-efficacy for scientific observation skills influences goal setting 
and task strategies in the forethought phase. During this stage, individuals will plan 
how to collect data on a specific topic such as a living organism. The beliefs an 
individual has about their capabilities in analyzing the organism influence how the 
individual plans to procure information about the topic of study. In the following 
phase (performance], a model or guide is essential for steering accurate 
observations. The facilitator provides demonstrations and gives feedback as the 
individual begins to collect data. Without this assistance and high self-efficacy, an 
individual is likely to rely on maladaptive strategies limiting their cognitive 
performance. Specifically, they may not notice the nuances which distinguish 
organisms in terms of classification, age, gender, or distinctive anomalies (Eberbach 
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& Crawley, 2009], In the self-reflection phase, individuals gauge their performance. 
Without strong self-efficacy beliefs related to scientific observation skills, rather 
than reflecting on how to improve their skills, a student may attribute unfavorable 
outcomes to perceived lack of instruction, the environment, or other uncontrollable 
influences. Inability to reflect on individual strategies rather than on external 
influences may result in a lack of motivation to improve in this cyclical process 
(Bembenutty, Kitsantas, & Cleary, 2013] and engage in the science oriented activity. 
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Figure 1 . Self-efficacy for scientific observation skills within a cyclical, self- 
regulatory feedback loop 


The implications of fostering self-efficacy have spurred the development of 
measures which assess task specific self-efficacy beliefs. For example, Zimmerman 
and Kitsantas (2007] produced a self-efficacy for self-regulation scale known as the 
Self-Efficacy for Learning Form [SELF] measuring student self-efficacy beliefs about 
their ability to adjust within a variety of educational structures. Aside from 
establishing high validity and reliability, findings showed that the 57-item scale 
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based on self-efficacy served as a strong predictor of grade point average, student 
perceptions of responsibility for academic performance, and homework completion 
in terms of consistency and quality. 

In terms of science achievement, Britner and Pajares (2006] focused on creating 
a self-efficacy scale targeting the four sources of self-efficacy, originally described by 
Bandura [1997], Britner and Pajares designed the measure to include mastery 
experiences, vicarious experiences, social persuasion, and physiological states. 
Mastery experiences describe an individual's perceptions of their level of expertise 
in a specific area and of the four sources is the greatest predictor of academic 
performance. Vicarious experiences relate to the influence of models and/or 
mentors on an individual; social persuasion derives from verbal and nonverbal 
feedback from peers and adults which shape beliefs related to learning, and anxiety 
is a physiological state which can be negated through overall heightened sense of 
self-efficacy (Bandura, 1997], Britner and Pajares concluded that the Sources of 
Science Self-Efficacy had high reliability for each of the four subscales. Notably, the 
scale of mastery experiences had a higher predictive value over the other three 
sources in terms of determining academic performance. 

Science education researchers have noted that there is a relationship between 
self-efficacy, science achievement, and career paths (Navarro, Flores, & 
Worthington, 2007; Patrick, Care, and Ainley, 2011], For example, self-efficacy may 
be extended to decision making and self-motivation beliefs related to STEM career 
paths. The Middle School Self-efficacy Scale is an example of an instrument which 
measures math and science self-efficacy, as well as career goals and intentions, 
outcome expectations, and interest (Fouad, Smith, & Enochs, 1997], According to 
Bandura, since self-efficacy is context specific, researchers should strive to create 
measures which address subtleties of relevant tasks. Although self-efficacy has been 
measured within science domains, there has been less emphasis on self-efficacy for 
scientific observation skills. 

Given that much of the literature on citizen science focuses on measuring science 
literacy (Cronje, Rohlinger, Crall, & Newman, 2011; Sutton, 2009], and the accuracy 
of volunteer data collection (Crall, Newman, Stohlgren, Holfelder, & Graham, 2011; 
Fowler, Whyatt, Davies, & Ellis, 2013; Gardiner, Alle, Brown, Losey, Roy, & Smyth, 
2012; Pocock & Evans, 2014], the purpose of the present study is to focus on 
assessing student self-efficacy and establish the validity and reliability of the Citizen 
Science Self-Efficacy Scale (CSSES] scale. Research suggests that establishing self- 
efficacy for scientific observation skills promotes stronger self-regulatory processes 
and achievement in natural science learning environments and promotes science 
oriented career paths (Hiller & Kitsantas, 2014]. Citizen science offers a distinct 
platform for training children in outdoor environments as data collection protocols 
often require modeling and guidance from field experts. Valid and reliable self- 
efficacy measures may enable researchers to highlight the positive influences of 
pedagogical approaches both in formal and informal learning settings. As self- 
efficacy scales are dependent on contextual settings, the purpose of this work is to 
describe the process of developing a self-efficacy scale which may be used to 
highlight the impact of outdoor programs on student learning. 

METHOD 

There are three key steps in the development of a self-efficacy scale: (a] 
consulting experts in the field and piloting the measure based on expert feedback, 
(b] revising the measure with exploratory factor analysis, and (c] testing the validity 
of the measure through confirmatory factor analysis. As an example, this work 
describes the steps taken to create the Citizen Science Self-Efficacy Scale (CSSS] 
based on a horseshoe crab citizen science program for middle school students. 
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Three studies were conducted to develop the Citizen Science Self-Efficacy Scale 
(CSSES] and evaluate its psychometric properties (see Figure 2], The first study 
framed the process of item development based on the experience of professional 
field experts who work with children. Items were also piloted with middle school 
children. In the second study, an exploratory factor analysis was conducted to 
examine the construct validity and the internal consistency of the scale. Finally, the 
third study confirmed the structure of the scale and concurrent validity with other 
related scales 


■ Pilot initial items 
■Field experts (n =15) 
and middle school 
students (n = 12) 


Phase 1: Initial Item 
Development 


Phase 2: Exploratory 
Factor Analysis 


■ Examine the construct 
validity and internal 
consistency of the 
CSSES 

■ Middle school 
students (n= 113) 


• Report confirmatory 
factor analysis and 
concurrent validity of 
the CSSES 
■Middle school 
students (n =123) 


Phase 3: 

Confirmatory Factor 
Analysis 


Figure 2. The development of the Citizen Science Self-Efficacy Scale (CSSES] 


Instrument development: study I 

The initial study which steered the phases of development for the CSSES was a 
qualitative study on field experts' perceptions of fostering environmental education 
for children. Findings from this study informed subsequent item selection. In the 
next two studies, the revisions of the CSSES centered on emphasizing scientific 
observation skills, self-efficacy, and competence via mastery experiences through 
citizen science programs. 

Participants and setting 

Participants for the first study included 15 (n = 15] field experts. The criteria for 
participant selection were professional field experts, trained naturalist volunteers, 
and practitioners. Recruitment of these individuals initially began with purposive 
sampling (roger, 2002] through a naturalist organization. All of the participants had 
worked with children in outdoor education settings. In an effort to capture multiple 
perspectives based on geographical location and differing demographic 
backgrounds, participants were subsequently contacted through network sampling 
(Patton], Fifteen individuals participated in interviews across a 270 mile range in 
suburban (nine], urban (three], and rural (three] areas. Of these participants, eight 
were male and seven were female. The racial/ethnic background of the participants 
included African American (three], Hispanic (one], and White (eleven] In terms of 
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affiliations with environmentally based organizations, five of the participants were 
volunteers, six worked for federal agencies, two for nonprofits, one for a private 
organization, and one worked for county government. Aside from working with 
children, the participants were professional scientists in a variety of fields including 
geology, anthropology, entomology, ichthyology, and botany (Hiller & Reybold, 
2011 ], 

In addition to the field experts, twelve (n = 12] eighth grade students from a 
public school in the northeastern part of the United States volunteered for a citizen 
science program. Participants included five male and seven female students; African 
Americans [one], White [ten], and Asian [one]. Students collected data on horseshoe 
crabs for a professional scientist at three sites (national park reserve, a beach, and a 
naturalist center]. Participants completed surveys and interviews at the national 
park reserve. 

Data collection 

Initially, three field experts participated in semi-structured interviews. The 
interview centered on examining the participants' perceptions of based on effective 
educational practices in outdoor settings. These initial interviews led to an analysis 
approach known as constant comparative analysis (Corbin & Strauss, 2007] in 
which tiered phases of coding revealed emerging themes from participant 
viewpoints. Labels applied to these first three interviews established subsequent 
questions for the second phase known as axial coding. The next 12 participants were 
asked questions such as "What are key characteristics children need in 
environmental education programs?" and "How does learning in the outdoors differ 
from the school setting?" Based on subsequent coding during the axial phase, 
categories development resulted in three emergent themes; self-efficacy, scientific 
observation skills, and competence. To establish validity of individual accounts, 
participants subsequently volunteered to review transcripts as a form of member 
checking. 

Both the field experts' descriptions and the Sources of Science Self-efficacy 
(Britner & Parajes, 2006] framed a subsequent pilot measure targeting self-efficacy 
for scientific observation skills. Items were developed based on Bandura’s (1997] 
guidelines, and existing self-efficacy scales (i.e. Sources of Science Self-efficacy 
Scale], Then experts in science research and pedagogy reviewed these items. Twelve 
children (n =12] were also asked to respond and comment on the items that were 
created with the guidance of the field experts. These items focused on asking 
children how confident they were in using observation skills. For example, 
participants answered items such as how confident they were to "distinguish 
between male and female horseshoe crabs," "distinguish horseshoe crabs from other 
animals," and "collect data for a scientist's research study." 

RESULTS 

Findings revealed that field experts placed great emphasis on scientific 
observation skills and student motivational beliefs. By far, all of the naturalists 
emphasized the need to develop scientific observation skills in the outdoor through 
modeling, mentoring, and varied experiences regardless of their occupations, 
geographical locations, gender, or racial/ethnic background. In addition, an 
individual's perception of their capabilities was central in fostering student 
experiences in the outdoors. In particular, field experts focused on the need to 
motivate students and encourage an appreciation for the outdoors through repeated 
educational experiences. The emergent themes from study one, rooted in field 
expert perspectives, steered the design of a pilot study in which the CSSES targeted 
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self-efficacy for scientific observation skills within a citizen science program. 
Furthermore, feedback and comments provided from the children participants led 
to further item refinement and provided stronger evidence of content validity in line 
with the goal to capture self-efficacy for scientific observation skills during an 
authentic outdoor learning experience. 

Exploratory factor analysis: study II 

Following this initial phase of item development, the scope of the second study 
was to examine the construct validity and internal consistency of the CSSES. 

Participants and setting 

One hundred and thirteen students [n = 113] were recruited from two middle 
schools for this study. Students received a letter asking them to participate through 
the middle school science classroom and their names were entered in a drawing for 
a gift certificate. Consent was obtained from parents and students. The average age 
of the students was 13.37 years (n = 113, M = 13.37, SD = 1.45] while 40% were 
male and 60% were female. The racial/ethnic background of these students 
included 1% Asian, 30% African American, 4% Hispanic, 57% White, and 8% 
identified themselves as "Other." 

RESULTS 

Based on exploratory factor analysis, CSSES items were revised and administered 
in the third phase, which resulted in eight items, one factor loading, and a 
Cronbach's alpha coefficient for internal reliability of .90. In addition, the factor 
accounted for 58.18% of the variance with an eigenvalue of 4.65. The range of factor 
loadings was between .66 and .88. Slight changes to wording such as substituting the 
word "Distinguish” with "Find" in "Find differences between horseshoe crabs" and 
subsequent analysis resulted in one scale with eight items prior to the third study. 
Ultimately, in study three, an additional item "using a tape measure" was removed 
due to a low factor loading prior to conducting confirmatory factor analysis 

Confirmatory factor analysis: study III 

The purpose of study three was to conduct a confirmatory factor analysis and 
assess the concurrent validity in comparison with the Sources of Science Self- 
efficacy (Britner & Pajares, 2006], task interest, outcome expectations, and career 
goal setting. Participants for study three completed a series of scales to analyze the 
CSSES with the goal to validate this measure. 

Participants and setting 

For the third study, participants (n = 123] from two middle schools in the 
northeast region of the United States completed the measures. Data collection 
occurred during science class. All students were in eighth grade between the ages of 
13 and 15 (M = 13.30, SD = 1.41]. The racial/ethnic background of the students 
included 1.9% Asian, 24% African American, 5.3% Hispanic, 59.6% White, and 9.1% 
Other. In all, 41.6% of participants were male and 58.4% were female. 

Measures 

Sources of Science Self-Efficacy Scale 

For study three, all four subscales of Britner and Pajares' (2006] Sources of Self- 
Efficacy were included. The Cronbach' Alpha Reliability Coefficient for each of the 
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subscales, validated with middle school students, were Mastery (.90], Vicarious 
(.80], Social Persuasion (.88], and Physiological States (.91]. For the present study, 
the Cronbach’s Alpha Reliability Coefficient was (.87, .71, .84, and .90] respectively. 
Participants completed items from each of the four subscales using a five point 
Likert-type scale ranging from "Strongly Agree to Strongly Disagree." A sample item 
from the Mastery subscale was "I will work as long as necessary to complete a 
difficult science activity." For Vicarious Experiences a sample included "Many of the 
adults I know have jobs that require a good understanding of science." An example 
of a Social Persuasion statement was "My friends tell me that I am good at science," 
and another for Physiological States included "Just thinking about science makes me 
feel nervous" (Britner & Pajares, 2006], 

Citizen Science Self-Efficacy Scale (CSSES) 

Revised from the initial Citizen Science Self-Efficacy Scale and study two, items 
centered on scientific observation skills such as counting, classifying, collecting, and 
measuring as described by Eberbach and Crawley (2009] and field experts in the 
phase one study (Hiller & Reybold, 2011], These items were based on a five point 
Likert-type scale. Participants responded to items on how confident they were that 
they could complete activities such as "I can write down things I see when looking at 
horseshoe crabs," and "measure the distance between the eyes of a horseshoe crab 
accurately." In a final exploratory factor analysis, "Use a tape measure" was removed 
due to a low factor loading resulting in a one factor, seven item scale with a 
Cronbach's alpha reliability coefficient of .87. 

Task interest scale (adapted from Zimmerman & Kitsantas, 1999) 

The purpose of this scale was to measure student interest in outdoor activities 
which ranged from (a] analyzing water samples, (b] bird watching, (c] measuring 
horseshoe crabs, (d] collecting seashells, and (e] drawing. In this study, interest was 
measured using a ranking scale. As a result, there is no reliability coefficient 
available. Students ranked their preference for each of these activities as mutually 
exclusive events. In the present paper, this scale established the validity of the CSSES 
by a correlation comparison. 

Citizen Science Outcome Expectations Scale (Hiller & Kitsantas, 2014) 

This scale, by Hiller and Kitsantas (2014], measures student perceptions about 
the outcome (benefits] of participating in a citizen science program and relates to 
interest as described by social cognitive career theorists. The six scale item includes 
items such as "Studying horseshoe crabs will help me improve my science skills." 
The initial pilot test of this scale yielded a Cronbach’s Alpha Reliability Coefficient of 
.88. For the present study, the Cronbach's Alpha Reliability Coefficient was .90. 

Career Related Goals Scale (Mu, 1998) 

Goal setting serves as an underlying foundation of career trajectories. Mu’s 
(1998] Career Goals Scale assesses goal setting strategies situated in career 
planning. A sample item is "I am clear about the steps I need to take to achieve my 
occupational/career goals." Mu reported a Cronbach's alpha reliability coefficient of 
.92 with high school students. In a subsequent high school study, for two separate 
treatment groups was .86 and .89 (Roger & Creed, 2011] whereas a study with 
middle school students indicated the internal reliability as .90 (Hiller & Kitsantas, 
2014], For the validation study the Cronbach’s Alpha Reliability Coefficient was .86. 
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RESULTS 

Analyses of factorial validity and reliability 

The Citizen Science Self-Efficacy Scale (CSSES] is a seven item measure of student 
perceptions of their capabilities in scientific observation skills. An exploratory 
principal component analysis based on seven items instead of eight indicated that 
these factors accounted for 59.15% of the variance with an eigenvalue of 4.14. There 
was one factor with seven items with loadings ranging between .74 and .82 as 
shown in Table 1. The Cronbach's alpha reliability coefficient was .87. One item from 
the phase three scale was removed, "Use a tape measure accurately" due to some 
weak factor loadings. 

Confirmatory factor analysis 

A confirmatory factor analysis (CFA] was conducted to examine the factorial 
validity of the seven item CSSES latent variable. Prior to conducting confirmatory 
factor analysis, the assumption of multivariate normality for each observation was 
examined. Using SPSS, each observation was tested for possible limitations in 
conducting confirmatory factor analysis based on skewness and kurtosis. Results 
indicated that all data fell below acceptable cut off scores for skewness (< 2.0] and 
kurtosis (< 7.0] (Dimitrov, 2012], 

Comparative fit indices were examined for the goodness-of-fit (GFI] > .95 the 
comparative fit index (CFI] > .93 (Hu & Bentler, 1999], and the Tucker-Lewis index 
(TLI] > .93 (Tucker & Lewis, 1973], A value of less than .08 for the standardized 
mean square residual (SMSR] was considered for the model fit. In addition, the Root 
Mean Square Error of Approximation (RMSEA] with a value less than .05 and a 
confidence interval between .00 and .08 (Steiger, 1990] served as an additional 
indices of goodness of fit. 

To test the CSSES, a confirmatory factor analysis (CFA] was conducted to examine 
the factorial validity of the CSSES latent variable. Using AMOS 22.0 software, the 
global fit of the hypothesized latent variable was assessed through maximum 
likelihood estimation. Support for a one-factor solution (see Figure 3] was 
established as the following fit statistics suggested adequate fit: y 2 (14] = 18.09, p = 

.20; CFI = .98; TLI = .96; RMSEA = .06, 90% Cl = .00, .12 (Hu & Bentler,1999], 

Table 1. Factor Loadings of the Exploratory Factor Analysis of the Citizen Science Self-Efficacy Scale 


Scale Item Factor Loading 

Rate how well you can conduct the activities listed below: 

Locate horseshoe crabs .74 

Count horseshoe crabs .74 

Write down things I see when looking at horseshoe crabs. .82 

Find differences between horseshoe crabs .82 

Measure the distance between the eyes of a horseshoe crab accurately .74 

Distinguish horseshoe crabs from other animals .75 

Collect data for a scientist's research study .76 


Concurrent validity 

Correlation measures related to sources of science self-efficacy, interest, and 
career goal settings addressed the predictive validity of the CSSES as shown in Table 

2 . 
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Table 2. Pearson correlations among self-efficacy, interest, outcome expectations and career goals 


Variable 

Cronbach’s a 

a 

b 

c 

d 

2 

3 

4 

l.Sources of Science Self-Efficacy 









a. Mastery 


.87 

1.00 







b. Vicarious 


.71 

.63** 

1.00 






c. Persuasion 


.84 

.80** 

. 68 ** 

1.00 





d. Anxiety 


.90 

-. 68 ** 

-.51** 

-.58** 

1.00 




2. Task Interest 


- 

.12 

.30** 

.24* 

-.09 

1.00 



3. Citizen Science Self- 


.88 

.33* 

.30** 

.40** 

-.24** 

44 ** 

1.00 


Efficacy 

4. Citizen Science Outcome 


.90 

.26** 

.37** 

.41** 

-.12 

.47** 

.25** 

1.00 

Expectations 

5. Career Goals 


.86 

.27** 

.17 

29 ** 

-.17 

.14 

.18* 

.05 


Note. * = p < .05, ** = p < .001 



Figure 3. Confirmatory factor analysis of the Citizen Science Self-Efficacy Scale 
[CSSES] 

In general, there were significant correlations among all variables with the 
exception of Career Goals and Interest [r = .14], and Career Goals and Outcome 
Expectations [r =.05], The lack of a relationship with these scales may be due to that 
the career goal questions included broad questions related to career planning that 
are not science specific. The strongest correlations were between Mastery 
Experiences and the other three sources of science self-efficacy; Vicarious [r =.63], 
Social Persuasion [r =.80], and Anxiety [r = -.68], Further the CSSES corresponded 
with the sources of self-efficacy-mastery(r =.33], vicarious(r =.30], social persuasion 
[r =.40], and physiological states [r = -.2], and most strongly with interest [r =.44], 
Similarly, there was a significant relationship between the outcome expectations 
scale and mastery experiences [r =.26], vicarious experiences [r =.37], social 
persuasion [r =.41], and interest (r =.47] as well as between the CSSES and the 
Citizen Science Outcomes Expectations [.25], In line with social cognitive career 
theory, the findings of this study reveal significant correlations among self-efficacy, 
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interest, outcome expectations, and goal setting. Corresponding with literature on 
adolescent achievement, the strongest relationship in this study was between social 
persuasion and mastery experiences. For students in secondary school, peer 
interactions are essential in the development of science achievement and self- 
efficacy status. 

DISCUSSION 

The present study examined the psychometric properties of the CSSES. The 
findings show that CSSES had an acceptable unitary factorial structure and high 
internal reliability of .89. Predictive validity indicated that there were significant 
correlations for previously established measures including the Sources of Science 
Self-Efficacy (Britner & Pajares, 2006] and Mu's Career Goal Scale (1998], Further, 
from a methods standpoint, the contribution of this work is to serve as a guide of 
how to develop a self-efficacy scale. In this work, the children studied horseshoe 
crabs, and the scale focused on the tasks required to collect data on the organism. 
Although, the CSSES may not have wide applicability to all citizen science programs, 
the contribution of this work is to help researchers produce their own citizen 
science self-efficacy scale. 

Since self-efficacy is a strong predictor of academic achievement, researchers 
may develop and use self-efficacy scales to examine the influence of outdoor 
learning on students' motivational and cognitive growth. In this work we outlined 
how a series of studies resulted in a citizen science self-efficacy scale. Initially, 
experts were consulted through qualitative methods to create an initial self-efficacy 
scale. A pilot scale based on their feedback was administered to middle school 
students. In the next phase, items were revised, administered, and examined 
through exploratory factor analysis. In the last study, the validity of the items was 
tested through confirmatory factor analysis. Implementing a series of studies in this 
way, may yield valid measures for subsequent studies. 

In recent years, citizen science research has shifted to examine the impact of 
authentic real-world experiences on student achievement and career motivation. 
With the onset of new and affordable hand held technologies, students are able to 
collect accurate information in the environment with sensors and probes. Access to 
accurate equipment may make it possible to involve adolescent students in research 
activities with minimalized concerns over inaccurate measurement collection. In 
addition, future citizen science programs may offer opportunities for students to 
interpret results in addition to being involved in data collection. One way for 
researchers to study the impact of these types of endeavors with school aged 
children is to develop self-efficacy scales. 

Applications for future research include testing the CSSES in horseshoe crab 
citizen science programs, applying the scale to other age groups, using the measure 
for longitudinal studies, or generating new self-efficacy scale for studying alternate 
biotic and abiotic topics. To our knowledge, this is one of the first studies to develop 
a measure which is applicable to social cognitive career theory within an informal 
natural science learning environment. Subsequent studies which incorporate 
researcher developed self-efficacy scales with larger numbers of students may yield 
useful information in promoting science achievement and career motivation within 
outdoor learning environments. 

RECOMMENDATIONS FOR PRACTICE 

In this article, we described the series of studies used to develop a self-efficacy 
scale which was applicable to a citizen science program. The purpose of the Citizen 
Science Self-Efficacy Scale (CSSES] is to assess individual's beliefs about their 
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capabilities for scientific observational skills. This scale is applicable to measuring 
individual's self-efficacy in outdoor learning contexts (e.g., horseshoe crab citizen 
science context]. The CSSES may be adaptable to other citizen science programs 
which aim to study self-efficacy for scientific observation skills with adjustments for 
the topic of study. The development of this type of self-efficacy scale may provide 
valuable information for researchers studying the impact of informal learning 
experiences on children's self-regulatory and motivational beliefs. For individuals or 
organizations which are promoting outdoor education, an obstacle for 
substantiating funding for outdoor excursions for school aged children is the need to 
establish the positive benefits of this type of activity. Given that self-efficacy is a 
strong predictor of academic achievement and motivation, self-efficacy scales like 
the CSSES may provide a way for stakeholders involved in outdoor education to 
measure student gains and to substantiate program effectiveness. 

Citizen science projects have the potential to provide meaningful scientific 
experiences for students as the relevance and significance of previous citizen 
science projects can be used to highlight real world implications of these projects. 
For example, in the UK, data from citizen science projects surveying bird 
populations in British gardens has been used to inform conservation monitoring 
(Cannon, Chamberlain, Tomas, Hatchwell, & Gaston, 2005] A recent citizen science 
project measuring trends in bat populations in the UK captured "Red Alert Levels" 
declines in bat population, a finding which could trigger environmental policy 
changes (Barlow et al., 2015], Finding like these can help inspire environmentally 
conscious students towards scientific careers given the potential for making positive 
changes as the result of involvement in citizen science programs. 

Moreover, while students learn to work on collaborative teams during authentic 
experiences, the development of self-efficacy for scientific observation skills 
supports student science development and career aspirations. Citizen science 
programs geared for middle schools, in conjunction with opportunities to work with 
field experts, positively impact student self-motivational beliefs such as self-efficacy 
as well as promoting cognitive development and STEM career motivation (Hiller & 
Kitsantas, 2014], In fact the Center for Advancement of Informal Science 
Information (Sako, 2015] cites the need for increased measures to highlight the 
impact of citizen science on individual development. 

In terms of science achievement and career motivation both in formal and 
informal science undertakings, providing opportunities to foster student growth in 
scientific observation skills is an essential element in science oriented career 
endeavors. Programs aiming to immerse children in meaningful activities which 
foster mastery experiences and self-efficacy, particularly through modeling and 
mentoring, align with current understandings of promoting student science 
performance and career growth. As such, establishing the psychometric properties 
of self-efficacy instruments such as the Citizen Science Self-Efficacy Scale (CSSES] 
serve as viable means for highlighting student progress and supporting funding 
opportunities. 

CONCLUSION 

Self-efficacy scales are a way to measure the impact of educational programs on 
student self-motivational beliefs. A process which includes consulting experts, 
administering and revising items based on exploratory factor analysis, and 
establishing validity through confirmatory factor analysis, are pivotal phases needed 
to generate this type of scale. The purpose of the CSSES was to develop and validate 
a measure suitable for use within a social cognitive career framework and informal 
natural science setting, specifically in studying a biological organism. Findings 
revealed that CSSES showed acceptable internal consistency and good construct and 
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predictive validity. This type of scale may be useful in helping educators assess the 
effectiveness of their citizen science programs in fostering student self-efficacy, a 
key predictor of behavior and learning. More importantly, the description of the 
formation of the CSSES may assist researchers in creating context specific measures. 
As a greater demand in promoting informal learning opportunities arises, 
researcher developed self-efficacy scales may prove to be a useful tool in 
highlighting the advantages of outdoor experiences. The methods of this work serve 
as a model for future endeavors. For example, subsequent studies may center on 
creating a battery of scales which include outcome expectations, interest, and career 
goal setting. Understanding student beliefs about their capabilities in a task, the 
value of the task, and goal setting are integral components in task mastery and 
cognitive growth. The studies described in this article, outline how to create 
measures which highlight student progress. The relationship between self-efficacy 
and these constructs with well-constructed measures may assist researchers in 
assessing the effectiveness of citizen science activities through new designs. 
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